Video signal post-processing method

ABSTRACT

The present invention relates to a method of post-processing pixels contained in a sequence of digital images. Said method comprises a step ( 200 ) of detecting pixels belonging to natural contours ( 20 ) inside an image. It also comprises a step ( 210 ) of detecting an investigation zone ( 21 ) corresponding to a coding block. It comprises a filtering decision step ( 220 ) such that a current pixel is filtered if it has not been detected as being a natural contour pixel ( 20 ) and if it belongs to an investigation zone ( 21 ) containing at least one natural contour pixel ( 20 ). Finally, the method comprises a pixel filtering step ( 230 ), of the median filtering type, for the pixels to be filtered.

FIELD OF THE INVENTION

The present invention relates to a method of post-processing pixelscontained in a sequence of digital images, said method comprising a stepof detecting pixels belonging to natural contours inside an image and apixel filtering step.

It also relates to a device implementing such a post-processing method.

The present invention in particular finds its application in the fieldof video coding. The coding technique is based for example on the MPEG(from the English “Moving Pictures Expert Group”) standard or anequivalent standard, by virtue of which a sequence of digital images ispreviously coded and then decoded in the form of blocks of data, thepresent invention allowing the correction of the data included in thedecoded sequence of digital images in order to attenuate the visualartifacts caused by the block-based coding technique. Thus it canadvantageously be integrated in video decoders or in televisionreceivers.

BACKGROUND OF THE INVENTION

The coding of an image sequence at low bitrate, using the MPEG codingtechnique or equivalent techniques, introduces visual artifacts into thedecoded images. Amongst the most usual artifacts there can first of allbe cited blocking artifacts which result in a visible division of theimage into blocks, generally of 8×8 pixels. A second type of artifactconsists of ringing artifacts. These are multiple echoes of naturalcontours, this visual defect also being referred to as the Gibbsphenomenon. Since these artifacts can be a great nuisance, it isnecessary to correct them.

Though there exist many methods for correcting blocking artifacts, thereare on the other hand very few methods for correcting ringing artifacts.The international patent application WO 2001/24115 (internal reference:PHF99584) describes such a method for the post-processing of pixelscontained in a sequence of digital images intended to reduce ringingartifacts. This post-processing method comprises a step of detectingpixels belonging to natural contours inside an image using for exampleSobel filters.

It also comprises a step of deciding on the filtering of a current pixelaccording to the surroundings of said pixel. For this purpose, thepost-processing method is able to divide the image into zones of 4×4pixels, the current pixel belonging to a central zone, and the East,North, West and South zones adjacent to the central zone being takeninto consideration. Thus a current pixel in the central zone is filteredif the following three cumulative conditions are satisfied:

it does not belong to a natural contour,

the number of natural contour pixels present in the 5 zones is less thana first predetermined value Nmax,

the number of natural contour pixels present in each zone is greaterthan a second predetermined value Nmin.

Finally, the post-processing method comprises a step of median filteringof a pixel to be filtered thus determined, from a vicinity of saidpixel. The vicinity of current pixel comprises certain pixels amongst aset comprising said current pixel and the East, North, West and Southpixels which are adjacent to it, depending on the fact that some of saidadjacent pixels are natural contour pixels or not.

Such a post-processing method is however relatively complex to implementsince it requires having values of pixels belonging to 5 differentzones.

SUMMARY OF THE INVENTION

The aim of the present invention is to propose a method for thepost-processing of pixels contained in a sequence of digital images,which is more simple to implement.

To this end, the post-processing method according to the invention ischaracterized in that it also comprises a step of detecting aninvestigation zone corresponding to a coding block, a current pixelbeing filtered if it has not been detected as being a natural contourpixel and if it belongs to a coding block containing at least onenatural contour pixel.

Thus the present invention takes account of the values of pixelsbelonging to a coding block, generally of 8 lines of 8 pixels, ratherthan the values of pixels belonging to 5 different zones of 4 lines of 4pixels. The present invention is therefore able to make a filteringdecision from a smaller number of pixels more easily accessible sincethey are distributed over 8 lines instead of 12, which makes theimplementation of the post-processing method more simple, from a memoryaccess point of view.

In addition, the present invention stems from the following analysis.The ringing artifacts result from an intense quantization of thetransformed DCT (standing for “Discrete Cosine Transform”) coefficientsinside the coding block. Starting from this assumption, it is possibleto deduce that the ringing artifacts, which correspond as we have seenpreviously to an echo of a natural contour, can be found only in aninvestigation zone corresponding to a coding block where at least onenatural contour is present. The consequence of this analysis is that theinvestigation zones not containing any natural contour are not filtered,which could happen with the method of the state of the art where thefiltering decision did not take account of the coding blocks. Thepost-processing method according to the invention is thus simplifiedfurther since it processes fewer pixels by eliminating all the pixels inthe investigation zones which do not contain a natural contour. It isalso more effective because it takes account of the block-based codingtechnique.

The present invention also relates to the device implementing the pixelpost-processing method according to the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be further described with reference to examples ofembodiments shown in the drawings to which, however, the invention isnot restricted.

FIG. 1 depicts the functioning of a complete chain for processing adigital video signal,

FIG. 2 is a schematic diagram of the pixel post-processing methodaccording to the invention,

FIG. 3 depicts a vicinity of a current pixel to be filtered, and

FIG. 3 depicts the detection of a chrominance sample belonging tonatural contours from a luminance sample.

DESCRIPTION OF PREFERRED EMBODIMENTS

The present invention relates to a method for the post-processing ofpixels contained in a sequence of digital images. Said method isintended to improve the visual quality of the digital images when thesehave been previously coded and then decoded according to a block-basedcoding technique.

The post-processing method was developed in particular for a codingtechnique based on the MPEG-2 or MPEGA standard. It nevertheless remainsapplicable for any other block-based coding technique such as H.261,H.263 or H.26L for example.

FIG. 1 illustrates the functioning of a complete chain for processing adigital video signal comprising coded digital images (10). Said chaincomprises a video decoder (11) comprising a decoding module (12) andintended to transmit a decoded image (14) via a channel (13) to atelevision receiver (15) intended to display the digital image thusdecoded. A correction or post-processing device (16), incontradistinction to a preprocessing of the signal which would havetaken place before the coding of said signal, improves the visualquality of the digital image with a view to its display on the screen(17). The post-processing device is situated either at the output of thevideo decoder as depicted in dotted lines or at the input of thetelevision receiver as depicted in solid lines.

In a first embodiment, the post-processing method is implemented at atelevision receiver. The pixel post-processing method illustrated inFIG. 2 thus comprises the following steps:

First of all a step (200) of detecting pixels belonging to naturalcontours inside a decoded image (14). For this purpose, the detectionstep is based on a gradient filtering GF (201) preferably usingtwo-dimensional Sobel filters, a filter Sh in the horizontal directionand a filter Sv in the vertical direction. It will be clear however to aperson skilled in the art that other gradient filters can be used fordetecting natural contours. The Sobel filters used are as follows.${Sh} = {{\begin{bmatrix}{- 1} & {- 2} & {- 1} \\0 & 0 & 0 \\{1} & 2 & {1}\end{bmatrix}\quad{and}\quad{Sv}} = \begin{bmatrix}{- 1} & 0 & 1 \\{- 2} & 0 & 2 \\{- 1} & 0 & 1\end{bmatrix}}$

The Sobel filters are applied to the luminance component Y of the pixelsof an image, the detection step resulting in an image of horizontalfiltered data Gh=Y*Sh and an image of vertical filtered data Gv=Y*Sv.

Then a thresholding THR (202) is applied to each of the two maps offiltered pixels. All the pixels p(ij) whose total filtered luminancevalue G(ij)=Gh(ij)²+Gv(ij)² is greater than a predetermined thresholdvalue T1 have the final value 1, i.e. they are detected as being naturalcontour pixels (20), the others having the value 0. The predeterminedthreshold value T1 is preferably an empirical value determined in anabsolute manner from a certain number of sequences of digital imagestested, equal, for example, to 12000 for luminance values of between 0and 255. This thresholding can be refined as follows, a pixel beingdetected as a natural contour pixel if:

-   -   G[ij]>T1 AND    -   (    -   ((|Gv[ij]|≧|Gh[ij]|) AND (G[ij]≧G[ij−1]) AND (G[ij]≧G[ij+1]))    -   OR    -   ((|Gh[ij]|≧|Gv[ij]|) AND (G[ij]≧G[i−1j]) AND (G[ij]≧G[i+1j]))    -   )

The predetermined threshold value can also be determined in a relativemanner as being equal to T2, whose calculation method is morecomplicated:${T\quad 2} = \frac{\sum\limits_{i = 1}^{n}{\sum\limits_{j = 1}^{m}\sqrt{{{Gh}( {i,j} )}^{2} + {{Gv}( {i,j} )}^{2}}}}{n \cdot m}$

where n is the number of lines and m the number of columns in the image.

In addition, the detection step is able to take into consideration theenvironment of a natural contour pixel thus detected. Thus it comprisesa refinement substep (203) such that, if a vicinity comprising the 8pixels surrounding such a pixel contains no other natural contour pixel,then said pixel is no longer assimilated to a natural contour pixel.Thus an isolated natural contour pixel is not considered to be a truenatural contour pixel (20).

The post-processing method also comprises a step (210) of detecting aninvestigation zone (21) corresponding to a coding block. At thetelevision receiver, no information relating to the decoding isaccessible. It is therefore necessary to analyze the content of at leastone image in order to determine the position and size of theinvestigation zone corresponding to a coding block. Such a blockgenerally comprises 8 lines of 8 pixels in the case of the MPEG standardbut the investigation zone may have a different size after anyresampling of the image during decoding, taken by assumption hereinafterat a size of 8 lines of 10, 12 or 16 pixels in accordance with the mainresampling formats allowed by the MPEG standard. A simple adaptation canbe made to the method described below for taking account of otherformats.

The step of detecting the investigation zone is preferably based on themethod described in the International patent application WO 01/20912(internal reference: PHF99579). The step of detecting the investigationzone comprises a horizontal and vertical gradient filtering substep GF(211) for the luminance pixels y(ij) of a digital image or for a portionof a digital image in the sequence, where i and j correspond to theposition of the pixel in the image. The gradient filtering step uses forexample the previously mentioned Sobel filters. There are then obtainedtables of pixels filtered horizontally xh(ij) and vertically xv(ij),whose absolute value ABS (212) is taken in order to obtain xah(ij) andxav(ij).

The investigation zone detection step also comprises a substep ofdetecting blocking artifacts BAD (213). Thus a vertical blockingartifact is detected if: $\{ {\begin{matrix}{{{xa}_{h}\lbrack {n,j} \rbrack} > {{{xa}_{h}\lbrack {n,{j - 1}} \rbrack} + \frac{\overset{\_}{{xa}_{h}}}{2}}} \\{{{xa}_{h}\lbrack {n,j} \rbrack} > {{{xa}_{h}\lbrack {n,{j + 1}} \rbrack} + \frac{\overset{\_}{{xa}_{h}}}{2}}}\end{matrix}\quad{\forall{n \in \lbrack {i,{i + 7}} \rbrack}}} $

Likewise, a horizontal blocking artifact is detected if:$\{ {{{\begin{matrix}{{{xa}_{v}\lbrack {i,m} \rbrack} > {{{xa}_{v}\lbrack {{i - 1},m} \rbrack} + \frac{\overset{\_}{{xa}_{v}}}{2}}} \\{{{xa}_{v}\lbrack {i,m} \rbrack} > {{{xa}_{v}\lbrack {{i + 1},m} \rbrack} + \frac{\overset{\_}{{xa}_{v}}}{2}}}\end{matrix}\quad{\forall{m \in {\lbrack {j,{j + k - 1}} \rbrack\quad{with}k}}}} = 8},{10\quad{or}\quad 12}} $

An analysis step SCAN (214) then determines the size and position of theinvestigation zone corresponding to a coding block.

For this purpose, when a vertical blocking artifact is detected for acolumn j, the value vTab8j %8), vTab10(j %10) and vTab12(j %12) of 3vectors vTab8, vTab10. vTab12 comprising 8, 10 and 12 values isincremented, a % b or a modulo b being the operation, the result ofwhich is the remainder of the division of a by b. Likewise, when ahorizontal blocking artifact is detected at line i, the value hTab(i %8)of a vector hTab comprising 8 values is incremented.

In parallel and in order to determine the size of the investigationzone, a general counter is created and then incremented at the readingof each pixel of an image or portion of an image. It is set to zero whena vertical blocking artifact is detected. The principle of thedetermination is to know whether a current vertical blocking artifact isdistant from the last vertical blocking artifact of 8, 10 or 12 pixels.Thus, if the value of the general counter between two vertical blockingartifacts is equal to 8, 10 or 12, then one of the counters grid8,grid10 and grid12 relating respectively to a width of the investigationzone of 8, 10 and 12 pixels is incremented. The width k of theinvestigation zone then corresponds to the counter which has thegreatest value.

This indication gives the vector vTabk, k=8, 10 or 12, to be taken intoconsideration from amongst the three possible vectors, and the origin ofthe investigation zone is determined by seeking the maximum values inthe vectors hTab and vTabk.

Other methods of detecting the investigation zone are also possible suchas the one, for example, described in the European patent applicationn^(o) 1202577 (internal reference: PHFR000106).

The post-processing method then comprises a filtering decision step(220): a current pixel is filtered only if it has not been detected asbeing a natural contour pixel and if it belongs to an investigation zonecontaining at least one natural contour pixel.

Finally, the post-processing method comprises a filtering step (230):the pixels which satisfy the criteria set out above undergo a filtering.This filtering is preferably a median filtering. A mean filtering canalso be envisaged.

This median filtering is able to replace a current pixel with a pixelfrom its vicinity. The vicinity of the current pixel comprises certainpixels amongst a set of pixels illustrated in FIG. 3 and comprising saidcurrent pixel (30) and the East (31), North (32), West (33) and South(34) pixels which are adjacent to it.

If none of the pixels in the vicinity is a natural contour pixel, thenthe value Y0 of the current pixel is replaced with the pixel whose valueis the median MED amongst the five luminance values of the currentpixels, East, North, West and South in the vicinity.

However, such a replacement preferably takes place only if this medianvalue MED differs from the value Y0 of the current pixel only by a valueDmax less than a predetermined threshold, for example equal to 40 if theluminance values are between 0 and 255, that is to say in other wordsif:

Abs(MED-Y0)<Dmax, where Abs(x) is the function which gives the absolutevalue of x. Thus such a filtering makes it possible to avoid makingerroneous corrections.

In the contrary case, the value Y0 of the current pixel preferablyremains unchanged, in order to reduce the complexity of the method.Nevertheless it can also be envisaged opting for a median filtering of aset of pixels comprising the current pixel and some of the East, North,West and South pixels which are adjacent to it, depending whether theseadjacent pixels are natural contour pixels or not, as described in theInternational patent application WO 2001/24115.

In a second embodiment, the post-processing method is implemented at avideo decoder. Compared with the embodiment previously described at thetelevision receiver, the post-processing method can be improved andsimplified by using the decoding information accessible at the decoder.The post-processing method then comprises the following steps:

a step (200) of detecting pixels belonging to natural contours inside animage. As in the first embodiment, this natural contour detection stepis based on the use of Sobel filters Sh and Sv. In this secondembodiment, the predetermined threshold value T1, determined in anabsolute manner from a certain number of sequences of digital imagestested, is taken to be equal to 20000.

In addition, the chrominance component of a pixel is taken into accountin addition to the luminance component. For this purpose, the naturalcontour detection is not applied directly to the chrominance values ofthe pixels of an image but is deduced from the luminance values asfollows, illustrated in FIG. 4 in the case of an image format of 4:2:0where there is a chrominance sample U and a chrominance sample V for 4luminance samples Y. Thus a value A(47) of chrominance U or V (42) isdeduced from the corresponding 4 final values a (43), b (44), c (45), d(46) issuing from the pixel detection step (200) applied to theluminance (41) in the following manner:

A=a OR b OR c OR d

with a, b, c or d=1 for a pixel with a natural contour and 0 otherwise.

Thus a chrominance sample is a sample belonging to a natural contour ifat least one of the four luminance values which correspond to it is anatural contour pixel.

The following steps are then the same for the luminance and chrominancecomponents, namely:

a step (210) of detecting an investigation zone (21) corresponding to acoding block. At the video decoder side, this step is easily performedbecause the coding blocks are directly accessible and therefore nolonger require the steps of gradient filtering GF (211), calculation ofabsolute value (212), detection of blocking artifacts (213) and analysis(214);

a filtering decision step (220): all the pixels belonging to aninvestigation zone containing at least one natural contour pixel andwhich are not natural contour pixels are intended to be filtered;

a filtering step (230): the pixels to be filtered undergo a filtering,preferably median. This filtering depends on the quantization step QP ofthe coding block to which the pixel to be filtered belongs.

If the quantization step QP is strictly less than a first predeterminedvalue Q1, no filtering is carried out, the quality of the coding beingjudged satisfactory.

If the quantization step QP is greater than or equal to the firstpredetermined value Q1 and less than or equal to a second predeterminedvalue Q2, a median filtering identical to that described in the firstembodiment is applied.

If the quantization step QP is strictly greater than the secondpredetermined value Q2, a strong correction must be applied. For thispurpose, a mean filtering is applied using the values of a set of pixelscomprising the current pixels and the East, North, West and South pixelswhich are adjacent to it.

Thus, if the pixel is not a natural contour pixel, its luminance valueY0 is replaced by the mean value Ymean:

Ymean=⅕*(Y0+Y1+Y2+Y3+Y4)

With:

Y1=Y(East) if the East pixel is not a contour pixel and Y1=Y0 otherwise,

Y2=Y(North) if the North pixel is not a contour pixel and Y2=Y0otherwise,

Y3=Y(West) if the West pixel is not a contour pixel and Y3=Y0 otherwise,

Y4=Y(South) if the South pixel is not a contour pixel and Y4=Y0otherwise.

Q1 and Q2 are values predetermined empirically, also respectively, forexample at 5 and 20 in the case of the MPEG4 standard, where thequantization step is between 1 and 31.

It is possible to implement the post-processing method according to theinvention by means of a video decoder circuit or a television receivercircuit, said circuit being suitably programmed. A computer programcontained in a programming memory can cause the circuit to perform thevarious operations described above with reference to FIG. 2. Thecomputer program can also be loaded into the programming memory by thereading of a data medium such as for example a disk which contains saidprogram. The reading can also take place by means of a communicationnetwork such as for example the Internet. In this case, a serviceprovider will make the computer program available to interested partiesin the form of a downloadable signal.

No reference sign between parentheses in the present text should beinterpreted limitingly. The verb “comprise” and its conjugations shouldalso be interpreted broadly, that is to say as not excluding thepresence not only of elements or steps other than those listed aftersaid verb but also a plurality of elements or steps already listed aftersaid verb and preceded by the word “a” or “one”.

1. A method of post-processing pixels contained in a sequence of digitalimages, said method comprising: a step (200) of detecting pixelsbelonging to natural contours (20) inside an image, a pixel filteringstep (230), said method being characterized in that it also comprises astep (210) of detecting an investigation zone (21) corresponding to acoding block, a current pixel being filtered if it has not been detectedas being a natural contour pixel (20) and if it belongs to aninvestigation zone (21) containing at least one natural contour pixel(20).
 2. A post-processing method as claimed in claim 1, in which thefiltering step is able to use a median filter able to replace a currentpixel with a pixel from a set comprising said current pixel and pixelssurrounding the current pixel.
 3. A post-processing method as claimed inclaim 2, in which a replacement takes place only if a median value ofthe set of pixels differs from the value of the current pixel only by avalue below a predetermined threshold.
 4. A post-processing method asclaimed in claim 1, in which the detection step is based on a gradientfiltering (201) using a filter of the Sobel type.
 5. A post-processingmethod as claimed in claim 1, in which the natural contour detectionstep (200) comprises a refinement step (203) such that, if a vicinitycomprising pixels surrounding a pixel detected as being a naturalcontour pixel, said vicinity containing no other natural contour pixels,then said pixel is no longer assimilated to a natural contour pixel. 6.A post-processing method as claimed in claim 1, in which the filteringstep is applied to a pixel in an investigation zone only if aquantization step for the corresponding coding block is greater than apredetermined value.
 7. A decoding method intended to provide decodeddigital images and comprising a post-processing method as claimed inclaim 1, in order to post-process the decoded digital images so as tosupply post-processed digital images.
 8. A device for post-processingpixels contained in a sequence of digital images, said devicecomprising: means for detecting pixels belonging to natural contours(20) inside an image, pixel filtering means, said device beingcharacterized in that it also comprises means for detecting aninvestigation zone (21) corresponding to a coding block, the filteringmeans being configured so that a current pixel is filtered only if ithas not been detected as being a natural contour pixel (20) and if itbelongs to an investigation zone (21) contains at least one naturalcontour pixel (20).
 9. A video decoder able to supply decoded digitalimages and comprising a post-processing device as claimed in claim 8,able to post-process the decoded digital images so as to supplypost-processed digital images.
 10. A television receiver able to receivedigital images and comprising a post-processing device as claimed inclaim 8, able to post-process the digital images so as to displaypost-processed digital images on the screen of the television receiver.11. A computer program comprising a set of instructions which, whenloaded into a circuit, causes the latter to perform the digital imagepost-processing method as claimed in claim 1.